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ABSTR ACT 



There is a need for clarity regarding different 
research strategies in order to assess the relative contribution of 
each. Using the framework of the Facet Design Technique (FDT) for 
problem structuring, eight general research strategies are identified 
as they relate to measurement fidelity, representativeness of units, 
and treatment administration* Three forces are seen as having 
contributed to a system of classifying research methods* They 
include: development of an instrument for evaluating the 
methodological adequacy of research in education; a growing reliance 
on the experimental method; and, the Gut.tman Facet Design Technique 
(FDT) which provides a system for the development of a conceptual 
universe on a specified problem in advance of the generation of 
empirical data. The historical, descriptive, experimental, and 
guasi-experi mental methods of research are defined in operational 
terms and classified into eight categories of research strategies 
through the use of the FDT. Levels of control, profiles of the 
different methods, and examples of less familiar inguiry techniques 
are examined. It is concluded that in some instances, data that have 
specific inadequacies can yield stronger conclusions through the 
application of certain statistical techniques, but that the overall 
evaluation of any completed research cannot be concluded until a 
contrast is made which holds up the statistic used to the statistic 
most appropriate for the nature of the generated data. (AE) 
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THE EIGHT GENERAL RESEARCH METHODOLOGIES: A FACET 

ANALYSIS OF THE RESEARCH PROCESS 

by 

William J. Gephart 
Director of Research Services 
Phi Delta Kappa, Bloomington, Indiana, U.S.A, 

Three forces have contributed to the system, for classifying research 
methods described in this paper. First, for several years the author has 
been working on the development of an instrument for evaluating the method- 
ological adequacy of research in education. Throughout this effort asser- 
tions have been made that one instrument cannot be expected to fairly 
evaluate the different research methodologies. Historical research, 
descriptive research, and experimental research are different entities 
according to these critics. Are they different, or are they facets of a 
single entity — the inquiry method? Until this is established, the 
question of the need for one or a variety of research evaluation instru- 
ments cannot be resolved. 

The second force is the growing concern for reliance on the experi- 
mental method. This concern is expressed by Guba in his discussion of the 
changing concept of research. He says. 

It appears there are two problems with the experimental 
approach: (1) it is not always possible or desireable to 

maintain the degree of control required by the laboratory- 
oriented experimental approach in a real-life setting, 
such as represented by the school, and (2) data from experi- 
ments do not answer all questions that might be asked. (1)* 

Debate on this topic has much emotional content. This writter has heard 



^Throughout this paper, references will be identified by their 
number on the References page. 
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dogmatic assertions to the effect that if experimentation is not present, 
research is not being done; that if an hypothesis is not being tested, 

fcHB tfOfck ia iftedftsequentiai; the respansei itm tne rebufIBB aBVMttfcHiT W 

historical and descriptive research are equally dogmatic. Again, until 
the similarities and differences of the several methodologies can be 
sorted, the argument cannot move off the emotional dead center of personal 
preference* 

Aligned with this concern is the confusion that ia exhibited in the 
literature about research methods. Typically, this term has referred to 
the three general strategies: historical, descriptive, and experimental. 

In recent years a fourth method, the quasi-experiment, has been introduced 
by Campbell and Stanley, (2) and a fifth suggested, the aexperimental 
method described by Guba (3). Even with the acceptance of these additions, 
some general inquiry approaches are left unclassified. For example, 
consider the case study approach, certainly not an experiment, perhaps 
somewhat a cross between a descriptive and a historical method. 

The third force leading to the eight method categorization presented 
hefe is the Facet Design technique developed by Guttman and described by 
Runkel (4). Guttman f s work provides a system for the development of a 
conceptual universe on a specified problem in advance of the generation 
of empirical data. Runkel* s discussion of his work shows how Facet 
Design can be used to conceptualize about the elements (facets) of a 
complex area before starting the empirical study, and how such an effort 
can improve subsequent investigations* This author and a colleague, 

Bruce Bartos, used this technique to provide a structure for research 
methods through which the eight general methods were identified. 

Before proceeding with the discussion of the use of Facet Design in 



the delineation of the eight general methods, it seems necessary to differ- 
entiate between the term “method" and the term "technique." The two 
are used interchangeably for two different meanings. At times they refer 
to the general research strategy, and at other times they mean the 
specific things done in a research project. This paper focuses on general 
research strategies and will use the term "method" in the belief that it 
is the broader of the two terms. Method in the discussion that follows 
will encompass techniques as the latter is used here to refer to specific 
things done in the generation and analysis of data. 

THE APPLICATION OF FACET DESIGN TO INQUIRY METHODS 

Runkel indicates that the activities involved in Facet Design are: 

1. The selection of elements known or suspected to comprise the 
problem area under scrutiny. 

2. The precise definition of those elements. 

3. The examination of those elements for common aspects (called 
facets) . 

4. The determination of the levels of those common aspects that 
can be seen in the problem. (Where possible these levels should 
be conceptual scales.) 

5. Listing the universe of profiles that exist through all possible 
combinations of the facet levels. 

6. Determination of the relationships among the universe of profiles. 
Each of these activities is illustrated as applied to inquiry strategy, 
and then the eight general methods will be discussed. 

Inquiry Methods Currently Described in the Literature and Their Definitions 



As Indicated earlier the literature on the research process refers 



to four general methods. These were selected for the starting point in 
the inquiry facet design. They are presented below, first with de- 
finitions taken from recognized texts on the research process. 

1. HISTORICAL METHOD — ". . . involves a procedure supplementary 

to observation, a process by which the historian seeks to test 
the truthfulness of the reports of observations made by others. 

Both the historian and scientist examine data, formulate hypoth- 
eses, and test the hypotheses against the evidence until accept- 
able conclusions can be drawn. " (5) 

2. DESCRIPTIVE METHOD — ". . , describes and interprets what is. It 
is concerned with conditions or relationships that exist; prac- 
tices that prevail; beliefs, points of view, or attitudes that 
are held; processes that are going on; effects that are being 
felt; or trends that are developing." (6) 

3. EXPERIMENTAL METHOD — ". . . a scientific investigation in which 
an investigator manipulates and controls one or more independent 
variables and observes the dependent variable or variables for 
variation concomitant to the manipulation of the independent 
variables." (7) 

4. QUASI- EXPERIMENTAL METHOD — "If the experimenter cannot or does 

not assign his experimental units at random to his experimental 
treatments, he performs other than a "true" experiment. . . . 

Intermediate between the frankly associational study and the 
experiment is a wide area called by CampbellL quasi-experimentation. 
. . . he does not have full manipulative control of his ex- 
perimental units." (8) 

Although these definitions are from recognized and reputable sources, 

o 
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they lack a precise delineation of the aspects of each method, a de- 
lineation necessary for Facet Design. After much discussion and consul- 
tation of the literature the following set of definitions evolved: 

1. HISTORICAL METHOD — is the determination of. truth about events, 
developments, and conditions of the past. It uses as measure- 
ments observations recorded by others to interpret 

to It involves the establishment of the popula- 

tion which experienced a set of events and the delineation of 
the nature of the experience. 

2. DESCRIPTIVE METHOD — is the determination of the manner in which 
a population is distributed on a variable or variables, and/or 
the degree of association among variables. It uses measures 
designed to validly and reliably collect the data. It focuses on 
a sp ecific sample and/or population because of things # that # may 

<?r # iji§Ly # n 9 p,h^ye # hfLppene<J to them. 

3. EXPERIMENTAL METHOD — is the determination of the cause and effect 
relationship among two or more variables. It involves the 
administration of specified^ treatments to a gojwl^aj^ion^oj^'^ 
sample of a population and the valid and reliable measure 

of the effects of the treatment. 

4. QUASI- EXPERIMENTAL METHOD — is the estimation of the cause and 
effect relationship among two or more variables in natural 
settings. It involves the administration of specif ied # treatments 
to an nn^ej^ec^t^d^gr^oi^ and the valid and reliable measure of 

the effects of the treatment. 

Facets of Inquiry: the Common But Variable Elements of the Definitions 



The third activity in a facet analysis is the examination of the 



definitions for common yet variable aspects. On the four definitions 



three items are common but variable, a thereof, 

measurement , and treatment. The historian is concerned about who^^^what^ 
is included in the phenomenon he studies. He has to establish the nature 
of the group on which he has obtained valid records and the nature of the 
population represented by this sample. The descriptive researcher, in his 
objective of depicting, has to ascertain the boundaries of the po^ul^at^ion 
he studies and the degree to which those on which he has measures represent 
this population. The experimenter must also be concerned about the £0£U— 

1 a. t icm and sample represen tat iv ene s s , for generalizations about cause and 
effect require a population referent. The quasi-experimenter is no less 
concerned about a Granted, he works in an arena in which he 

is denied direct control over representativeness. But he works in a 
natural setting because he wants to know about the effects of variables in 
that setting. Therefore, his inability to representatively select a 
sample and assign it to a treatment are not evidences of a lack of 
concern, but rather of his concern for reality. 

Similarly, the four methods involve treatments. This is an obvious 
factor in the experiment and the quasi-experiment. Although less obvious, 
it is a part of the historical and descriptive methods. The historian is 
either interested in determining the nature of a treatment experienced by 
a group or in the effects of a treatments The descriptive method focuses 
on a group for a reason. They have an apparently common set of experiences 
(treatment) which make them an interesting group for description. For 
example, descriptive studies have been done on the vocabulary possessed by 



children at various grade levels. We are interested in these children and 
their vocabulary levels for two reasons: we expect to administer instructive 
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treatments to them, and we feel sure that prior to the time they are 
studied they experienced things that affected their vocabulary. Despite 
our inability to describe these sets of events, they constitute treatments. 

Measurement is an obvious facet of descriptive, experimental, and 
quasi-experimental methods. In each case the investigator either selects 
an established means of measuring that will develop the required data, or 
he designs measures that will do the job. The historian is not so fortunate. 
He must use records made by others. Part of his task is the establishment 
of the credence of the recording source. A statement that a situation 
existed, made and recorded in the past, becomes his measurement. He must 
through a variety of means corroborate the validity and reliability of that 
statement. In so doing he evidences concern for measurement fidelity. 

Given the acceptance of these statements, three facets are clear: 
representativeness of the units studied, content of the treatment experi- 
enced, and measurement fidelity . Are there other items common to the four 
method definitions? Yes, each one either clearly states or implies that 
it is an effort to learn something. This is not considered as a facet, 
however, as it is a constant in all methods, rather than something that 
exists at different levels. 

Facet Variation: A Determination of Levels 

The next activity in the facet design technique is the determination 
of the levels of the various facets. In this, both Guttman and Runkel 
clearly display the advantage of looking for levels that are at least con- 
ceptual scales. They also imply that the levels might be determined by 
further examination of the definitions which serve as a basis for the analy- 
sis. As we reexamine the definitions, it is apparent that the descriptive 
and experimental methods are such that the investigator controls the 



representativeness of the units studied. On the other hand, the quasi- 
experimentalist and the historian do not have initial control. In coth 
of the latter cases, the selection of the units on which records have or 
are to be made is determined by someone other than the investigator. Thus 
there are at least two levels to this facet, under and not under the 
investigator’s control. 

There is a scale underlying these levels. If the item is under his 
control, the investigator can be expected to produce greater strength of 
conclusion. If it is not in his control, conclusions must be tempered 
with the possibility that undisclosed uniquenesses may cause the studied 
units to be different from the population in general. Thus, the level of 
this facet under-the-investigator ’ s-control is of greater strength than 
is the level not-under-control . 

The same pair of levels exists in the measurement fidelity and treat- 
ment facets. On the former, measurement fidelity facet, the historian 
lacks control. He uses observations recorded by others as his measures. 
The three other methodologies are instances in which the investigator 
decides what and how records should be made. On the treatment facet, 
the historian and descriptive researcher are in a non-control situation. 
They deal with units that have experienced a treatment, the elements 
of which the investigator cannot describe with surety. The experimenter 
and the quasi-experimenter, on the other hand, carefully structure the 
experience of the units studied to enable as strong as possible a state- 
ment about the relationship between treatment and consequence. (9) In 
both of these sets of levels the non-control case conceptually seems less 
productive of truth about an unknown than does the control. Thus, we have 
three facets, each of which can be analyzed into two levels, controlled or 
not controlled by the investigator. 
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The Universe of Profiles 

Such an analysis enables us to return to the original set of known . 
elements and profile them, part of the fifth step in Facet Design as stated 
by Runkel and Guttman. Profiling is accomplished by assigning numerical j 
values to the levels of the identified facets. In the case in point, norj- 
control is represented by the subscript 1 and control by 2. The historical 
method is an investigatory approach in which neither the measurement, repre- 
sentativeness, nor treatment are under the control of the researcher. He 
uses records made by others and kept on a sample that he was unable to j 
select originally as evidence of a treatment that he did not administer.! 
Thus, the profile for the historical method is m^ r p t r j 

Descriptive methodology is profiled as m 2 , r 2 , t^, since the researpher 
here has control over selection and measurement but not over treatment. (Ex- 
perimental method can be represented as m 2 > r 2 , t 2 > as the investigator jin 



this strategy has control over all three facets. The quasi-experimentalj 

method— since it is a strategy designed for those instances that do not jper- 

| 

mit the investigator control over the representativeness of his sample-j 
would be profiled as m 2 , r x , t ^ The table below indicates these profiles . 

I 

Facets and Levels ' 



Research 

Method 



Measurement 

Fidelity 



Representativeness Treatment 

of Units Administration 



Historical 


m i 


r l 


Descriptive 


m 2 


r 2 


Quasi-experiment al 


m 2 


r l 


■Experimental 


m 2 


r 2 



C 1 

t 2 

to 



An examination of these profiles leads to the conclusion that thefe are 
four types of profile sets. One set has all control level facet notation 
(the experimental method m 2 , r 2 , t 2 ) . Another has all non-control lev^l 
facet notation (historical method m^, r 1? t^). Between these extremes , is a 
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facet notation 



set that has two facets at the control level and one non-control (des- 
criptive and quasi-experiment al methods). The profiles for these two 
are m2, ^ > t^ and m2, r-p t2 respectively. There is a third possible 
profile that becomes discernable in this set, m-^, t2* This would be 

a strategy that involves careful control over treatment and representative- 
ness facets in a situation that does not permit control over measurement. 
Such a study might be labeled an unobtrusive-measurement-experiment. 

Having described three profile sets (all l's, all 2 *s, and two 2 *s 
and a 1 ) the fourth set becomes apparent. These profiles would be research 
strategies for situations in which it is possible for the investigator to 
exercise control over one of the facets but not over the other two. The 
profiles for such cases would be the three combinations of a 2 and two l T s 
( m 2 9 r^, t-^; m^, r2> t-^; and r^, t2)« This brings us to a total of 

eight profiles, all that possibly could exist if a system consists of three 
facets each with two levels. In effect this system is a 2 x 2 x 2 matrix 
which yields a total of eight separate categories. The table below lists 
the complete set, indicates the level of control on each of the three 
facets, and where possible supplies a descriptive name for the strategy. 

The material following the table will present briefs of studies that serve 
as illustrations of the heretofore unfamiliar strategies. 

Facets and Levels of Control 

Research Measurement Representativeness Treatment 

Method Fidelity of Units Administration 



A. 


Historical 


m l . 


r-i_ 


t. 


B. 




m 2 


r l 


x 

ti 


C. 


Case Study 


mi 


r 2 


tl 


D. 







ri 


t? 


E. 


Descriptive 


m 2 


r 2 


C 1 


F. 


Qua si- experiment 


m2 


r l 


t 2 


G. 


Unobstrusive- 
measure experiment 


- m ! 


. - £2 


t? 


H. 


Experiment 


m2 


r 2 


c 2 
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EXAMPLES OF THE LESS FAMILAR INQUIRY PROFILES 



In the interest of conservation of space the audience is asked to supply 
examples of historical, descriptive, quasi-experimental , and experimental 
research. Profile B (m2, r^, t-^) in which the investigator has control 
over measurement fidelity but not over the representativeness of the units 
studied or the treatment administration, is illustrated in an account of a 
study proposed by a professor as a cooperative dissertation for several 
doctoral students. The problem indicated an interest in determining the 
impact of higher education institutions on their urban surroundings. 

Impact in this case was defined as financial contribution and social 
attitudes and behaviors. Clearly, the investigator cannot control the 
treatment in this case. The college or university exists in the community 
and in varying and perhaps amorphous ways has an impact. The investigator 
has in reality a two-part task; he must determine what the higher education 
institution does and the impact of those actions. Further, if the things 
done by the institution are not known in advance, it is impossible to 
enumerate those to whom these things are done. Therefore, the researcher 
cannot assert that he has control over representativeness of units. 

As plans for this study have progressed, a nomination technique is being 
employed to define the population. That is, logically involved individ- 
uals are identified and interviewed. Persons mentioned in certain 
contexts in these interviews are to be added to the list of units studied 
and will by extrapolation define the population. Such a study has 
definite weaknesses; it can, however, be rigorously conducted and yield 
evidence on which to base future hypothesizing. There are individuals who 
will question the worth of the effort described above. Such individuals 
would perfer the comfort of the controlled experiment and by implication 



at least would defer any study until the facets of the investigation 
could be under control. If they succeed in such a demand, advancement of 
knowledge in areas such as this will be delayed for a long time. 

Research method C (m-^ , r^ , t-^) has been labeled in the table as a 
Case Study methodology. It represents a situation in which the total 
population is involved, a situation which should mean that control over 
the representativeness of the units studied is in the hands of the investi- 
gator. If a case study is being made of me, I am the best unit to study to 
insure that the study will yield sound conclusions about me. However, if 
you are going to conduct that case study, you have to use measures recorded 
by others and perhaps for other purposes. Finally, in your study of me, 
you must admit that you have no control over nor can you even describe the 
complex of treatments that have contributed to x^hat I am. Again there are 
individuals who would reject this as a valid research effort. We can 
learn from systematic case studies of individuals and institutions. 
Frequently such efforts facilitate the advancement of an individual or 
institution. Note here, there is no claim for broad generalizability of 
the findings of a case study. The results are of value and reliable for 
the unit studied, not for some general population. 

Method D (m-^, r^, t£) can be illustrated by reference to the reaction 
studies shown in the popular television program. Candid Camera. One 
program showed films taken of people in elevators. In each one there was 
one subject and a number of other elevator passengers working with the 
investigator. When the elevator stopped at a floor to pick up a passenger, 
all of the others faced the rear of the car. A film was made through which 
the reactions of the unsuspecting passenger was to be displayed. In such 
a study subjects are chosen quite by accident, thus not under the 



investigator's control. The measurement, a film record of the overt 
behavior, is an instance of taking what got recorded as the data. 

Numerous individuals failed to react overtly, thus making their segments 
of the film unusable as an entertainment medium. This does not say that 
they did not react. If such a study had been devised to examine reactions 
to group press, logic would dictate that a simple visual record would miss 
much of the effect. Thus, it is maintained that the case in point was 
one in which control over the fidelity of measurement was lacking. 

Treatment was controlled. In every case the cohorts of the investi- 
gator applied the group pressure. Is a study that lacks measurement and 
representativeness control of value? It would seem to be extremely so as 
a means of identifying some of the dependent variables operating in an 
unexplored area. 

The last method to be described by an example is one in which the 
representativeness and treatment facets are controlled by the investigator 
but the measurement occurs outside of his control (raj_ , r£, ^2^* Some of 
the research on reading instruction has asserted an interest in the degree 
to which children enjoy reading after receiving instruction in one manner 
or another. As an index of the enjoyment variable, library loan frequency 
has been suggested. In such a case the investigator has the possibility 
of control over sample selection and assignment and over the administration 
of the treatment, instructional methods. But, the data are generated by 
the library staff. The number of books checked out by the subjects in 
the study may or may not be an index of the degree to which the students 
enjoy reading as a result of the instruction they experienced. The 
substitution of a self-report on the degree of enjoyment has similar 
inadequacies, as again the investigator takes what the subject records. 
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SUMMARY 



Eefore summarizing it seems important to highlight the fact that 
this system, developed through the application of facet analysis to the 
inquiry process, does not contain a focus on the analytic techniques that 
must be applied to the generated data. It is recognized that in some 
instances data that have specific inadequacies can yield stronger con- 
clusions through the application of certain statistical techniques. The 
evaluation of any completed research cannot be concluded until a 
contrast is made which holds up the statistic used to the statistic most 
appropriate for the nature of the generated data. 

This paper has asserted that clarity about the nature of various 
research strategies is necessary before we can assess their relative 
contributions. Through describing the facet design technique for problem 
structuring, eight general research strategies are identified. The 
uncommon strategies were illustrated by examples. 



O 

ERIC 




- 14 - 



REFERENCES 



1- Guba, Egon G. , "The expanding concept of research." Theory 
Into Practice . VI: 57-65; April 1967. p. 60. 

2. Campbell, Donald T. , & Stanley, Julian C. , "Experimental and 
Quasi-Experimental Designs for Research on Teaching." in N. L. 

Gage, Ed. Handbook of Research on Teaching . Chicago: Rand 

McNally 1963. 

3. Guba, Egon G. , "Methodological Strategies for Educational Change." 

ERIC Document Number ED Oil 404. November 1965. 

4. Runkel, Philip, "Some Recent Ideas in Research Methodology." 

ERIC Document Number ED 010 221. 1965 

5. Travers, Robert M. W. , An Introduction to Educational Research: 

Second Edition . New York: The Macmillan Company, 1964, p. 183. 

6. Best, John W. , Research in Education . Englewood Cliffs, New Jersey: 
Prentice-Hall, Inc. 1959. p. 102. 

7. Kerlinger, Fred N. , Foundations of Behavioral Research . New York: 
Holt, Rinehart and Winston, Inc. 1964. p. 290. 

8. Stanley, Julian C. , "Quasi-Experimentation." The School Review 
73:197-205; 1965. p. 198. 

9. In another publication Gephart and Bartos have described numerous 

levels that exist on each of these three facets. Example: The 

Representativeness of Units facet ranges from perfect- representation- 
of- a- specif ied- population to a collection-of- units- were- studied 

and we know nothing about the sample they represent. Between these 
two extremes are samples that were volunteers, purposive sampling, and 
randomly selected subjects. The two extremes were used in the present 
paper for the considerations in facet design. For the expanded 
treatment see the final report on Cooperative Research Project 
Number 071018. Training Materials for Researchers, Developers, 
and Diffusers. (In progress, 1969.) 



